Evaluation of a behavioural intervention to reduce perioperative midazolam administration to older adults

Background Older patients commonly receive benzodiazepines during anaesthesia despite guidelines recommending avoidance. Interventions to reduce perioperative benzodiazepine use are not well studied. We hypothesized an automated electronic medical record alert targeting anaesthesia providers would reduce administration of benzodiazepines to older adults undergoing general anaesthesia. Methods We conducted a retrospective study of adults who underwent surgery at 5 hospitals within one US academic health system. One of the hospitals received an intervention consisting of provider education and an automated electronic medical record alert discouraging benzodiazepine administration to patients aged 70 years or older. We used difference-in-differences analysis to compare patterns of midazolam use 12-months before and after intervention at the intervention hospital, using the 4 non-intervention hospitals as contemporaneous comparators. Results The primary analysis sample included 20,347 cases among patients aged 70 and older. At the intervention hospital, midazolam was administered in 454/4,240 (10.7%) cases pre-alert versus 250/3,750 (6.7%) post-alert (p<0.001). At comparator hospitals, respective rates were 3,186/6,366 (50.0%) versus 2,935/5,991 (49.0%) (p=0.24). After adjustment, the intervention was associated with a 3.2 percentage point (p.p.) reduction in the percentage of cases with midazolam administration (95% CI: (-5.2, -1.1); p=0.002). Midazolam dose was unaffected (adjusted mean difference -0.01 mg, 95% CI: (-0.20, 0.18); p=0.90). In 76,735 cases among patients aged 18–69, the percentage of cases with midazolam administration decreased by 6.9 p. p. (95% CI: (-8.0, -5.7); p<0.001). Conclusion Provider-facing alerts in the intraoperative electronic medical record, coupled with education, can reduce midazolam administration to older patients presenting for surgery but may affect care of younger patients.

Few data exist as to how to discourage potentially avoidable perioperative benzodiazepine administration to older patients. One institution recently described a multi-component quality improvement initiative that decreased benzodiazepine administration to patients presenting for surgery. 10,11 Further investigation into simple and scalable interventions focused on de-adoption of routine benzodiazepine administration may help clinicians, policy makers, and health systems optimize care of older surgical patients.
Nudges are environmental modifications designed to improve decision-making 12,13 ; clinician-directed nudges, like automated electronic medical record (EMR) alerts, can promote guideline-concordant behaviours. 14e18 The ability of nudges to discourage routine perioperative benzodiazepine administration to older surgical patients remains poorly characterized.
We tested the impact of an automated, clinician-directed EMR alert, coupled with educational outreach, on perioperative administration of midazolam to adults aged 70 years or older undergoing elective non-cardiac surgery. We hypothesized that exposure to the alert would be associated with decreased rates of benzodiazepine administration to patients requiring general anaesthesia.

Methods
Overview This is a retrospective study of data from 5 hospitals in one academic US health system. One of the hospitals received a clinician-directed EMR alert discouraging benzodiazepine administration to surgical patients aged 70 years or older with educational outreach introducing the alert; the 4 remaining hospitals received no interventions and served as a grouped comparator. The effect of the alert on midazolam administration was estimated via difference-in-differences analysis (described below). 19e21 The University of Pennsylvania Institutional Review Board exempted this study from review and waived the requirement for participant informed consent.

Intervention
On April 4, 2019, a departmental conference focusing on potentially avoidable medications in older adults, such as benzodiazepines, was delivered to clinical anaesthesia staff at the intervention facility. This was immediately followed by a department-wide email summarizing the presentation. Selective administration and dose reduction were encouraged. On 17 th June, 2019, an automated EMR alert entitled "Delirium Prevention in Older Patients" was activated within the intraoperative anaesthesia record. The alert displays automatically for every anaesthesia provider who opens the intraoperative component of the anaesthesia record for patients aged 70 years or older. The alert text suggests avoidance of benzodiazepines with midazolam as the prototypical medication (Supplementary Figure 1). Also listed are other medication classes identified in American Geriatrics Society guidelines 3 as being potentially inappropriate for use in older adults with a link to the guideline text. No similar interventions occurred at comparator hospitals over this period.
Planning of the intervention occurred separately from planning of this analysis. We focused our evaluation on benzodiazepines based on their frequency of use and evidence suggesting lack of benefit with potential for harm in most older surgical patients. 5,7 Participants and data sources We obtained data on all patients aged 18 years or older who received general anaesthesia or sedation 22 for elective noncardiac surgical procedures between 18 th June, 2018 and 30 th June, 2020. For our main analysis, we defined the pre-alert period as including all cases performed between 18 th June, 2018 and 31 st May, 2019; we defined the post-alert period as including cases performed between 1 st July, 2019 and 30 th June, 2020. We excluded cases performed between 1 st June, 2019 and 30 th June, 2019 from the primary analysis to allow for a run-in period for alert implementation. The type of anaesthesia received was determined based on EMR documentation; we coded cases as general anaesthetics if they involved placement of a supraglottic airway or endotracheal tube. We excluded patients with ASA physical status classification 23 V or VI, emergencies, and cases where regional anaesthetics were administered before induction of general anaesthesia.

Outcomes
The primary outcome was a binary variable indicating the receipt of midazolam as recorded in the intraoperative anaesthesia record. We studied midazolam as non-midazolam benzodiazepines account for fewer than 1% of perioperative benzodiazepine administrations. 8,9 The dose of midazolam administered (when given) was a secondary outcome.

Independent variables
We obtained data from the EMR using Multicenter Perioperative Outcomes Group definitions 24 on patient age, sex, race (Black, white, or other), Hispanic ethnicity, ASA physical status classification, body mass index, and 9 comorbidities (dementia, anxiety, cerebrovascular disease, cardiovascular disease, hypertension, chronic obstructive pulmonary disease, kidney disease, diabetes, peripheral vascular disease), use of inhaled anaesthetics, procedure duration, and surgical department.

Statistical analysis
We performed a difference-in-differences (DID) analysis, a widely-used technique in economic research and policy evaluation. Our analytical plan was finalised in November 2020. Analysis began in May 2021. Unlike uncontrolled pre-post analyses, DID uses a contemporaneous comparison group (here, those hospitals that did not receive the intervention) to account for temporal changes in a particular outcome (here, midazolam use) that might otherwise confound causal inference (Supplementary Figure 2). 19e21 A key assumption of DID is that trends in the outcome prior to the intervention are similar for both intervention and comparison groups (the so-called "parallel trends assumption"). Importantly, estimates from DID analysis are still valid even if the level of the outcome (e.g., the rate of midazolam use) differs for intervention versus control facilities, as long as trends in outcome prior to the intervention are parallel across treatment and control facilities. 19e21 We used descriptive statistics and simple hypothesis tests to compare patient and procedure characteristics and outcomes at the intervention versus comparator hospitals. We fitted a linear probability model 25,26 to predict our primary outcome based on the interaction of exposure group (intervention versus comparator hospital) and period (pre-versus post-alert), plus controls for patient characteristics, comorbidities, and surgical categories as above; the coefficient of this interaction term in the regression model corresponds to the impact of the intervention on the study outcome. 19e21 We evaluated the parallel trends assumption via visual inspection and in a regression model estimating our outcome using data from the 12-month pre-intervention period only. This model predicted the primary study outcome along with an interaction between treatment arm (intervention versus comparator hospitals) and whether the case occurred during the first 6 months or the second 6 months of the preintervention period, plus all controls listed above. A nonsignificant estimate on the interaction term indicates parallel pre-intervention trends across the intervention versus control hospitals in the primary outcome.
To test the robustness of our findings, we repeated our main model to predict an endpoint of glycopyrrolate administration, a medication that was not specifically targeted by the alert. This served as a falsification test 19e21 ; a finding of no effect on glycopyrrolate administration could be taken to confirm that our findings were likely to be due to the targeted impact of the EMR alert on midazolam versus some broader unspecified change in perioperative medication administration patterns. In contrast, finding that the alert increased or decreased glycopyrrolate use could be taken to imply the opposite. We also re-estimated our main model with an indicator variable for each month of study (time fixed-effects) and each hospital (group fixed-effects) to directly control for trends in the outcome variable over time and unobserved heterogeneity across hospitals. Finally, to assess whether inclusion of more than one case per patient may have affected our main results, we repeated the primary analysis at the patient-, rather than case-level, using the first listed case for each patient.

Supplementary analyses
To assess whether the intervention affected younger patients, we fitted the same models in our secondary sample of patients aged 18e69 years who underwent general anaesthesia for elective non-cardiac surgery. We also analysed cases performed under sedation, since clinical decision-making for benzodiazepine use in this context may differ from that used for general anesthesia. 27 To separate the potential effects of the components of the intervention (education vs EMR alert) on outcomes, we fitted a DID model that separately estimated changes in the outcome variable during the "education-only" phase of the intervention (i.e. between 4 th April, 2019 and 16 th June, 2019) and during the period after EMR alert activation (i.e. between 17 th June, 2019 and 30 th June, 2020) compared to the period before 4 th April, 2019.

Data analyses and reporting
Analyses were performed using Stata 17.0 (Statacorp, College Station, TX). P<0.05 was our threshold for statistical significance. The strengthening the reporting of observational studies in epidemiology (STROBE) checklist appears in Supplementary Table 1.

Characteristics of the primary study sample
After exclusions (Supplementary Table 2), our primary analysis included 20,347 cases performed in adults aged 70 or older (range 70e90 years old). 10,606 (52.1%) of these cases occurred in the 12 months before alert implementation, and 9,741 (47.9%) occurred after implementation. Over the full study period, most patient characteristics were similar at the intervention and comparator hospitals (Table 1), although cases at the intervention hospital were longer, involved patients with higher ASA physical status scores, and less frequently employed inhalational anaesthetics. Endocrine procedures were more common at the intervention hospital (12.6% of cases at intervention versus 1.6% of cases at comparator). Orthopaedic procedures were more common at the comparator hospitals (1.0% at intervention versus 23.8% at comparator). Other procedure types were similar between locations.

Midazolam administration within the primary population
In the period before alert deployment, at the intervention hospital, midazolam was administered in 10.7% (454/4,240) of cases involving patients 70 years or older compared with 6.7% (250/3,750) of cases post-alert (p<0.001). In contrast, at the comparator hospitals, midazolam was administered in 50.0% (3,186/6,366) of cases versus 49.0% (2,935/5,991) of cases respectively (p¼0.24) ( Figure 1). Supplementary Figure 3 shows patterns of midazolam administration by month across individual study hospitals.
For cases at the intervention hospital, the mean midazolam dose per case was 1.69 mg (SD 0.68 mg) pre-alert versus 1.72 mg (SD 0.73 mg) post-alert (p¼0.59). At comparator facilities, the mean dose per case was 1.86 mg (SD 1.37 mg) versus 1.93 mg (SD 4.62 mg; p¼0.47) respectively ( Figure 1). After adjustment for patient and procedure-level factors, exposure to the alert was associated with a significant decrease in the percentage of cases in which midazolam was administered (difference-in-differences (DID) estimate -3.2 percentage points (p.p.); 95% CI (-5.2, -1.1); p¼0.002; Table 2). The midazolam dose administered did not change (DID estimate -0.01 mg, 95% CI (-0.20, 0.18 mg), p¼0.90). We obtained similar findings in models that included hospital fixed effects and month fixed effects (Supplementary Table 3) and in a sample that used only the first available procedure for each patient (Supplementary Table 4, Supplementary Figure 4).

Sensitivity analyses
The alert did not significantly impact the percentage of cases with glycopyrrolate administration for patients aged 70 years or older (DID estimate -0.  Figure 5B).

Discussion
Within one US academic health system, a clinician-facing, automated, EMR alert, preceded by clinician education, reduced the percentage of cases of general anaesthesia in which patients aged 70 years or older received midazolam by an adjusted 3.2 percentage points (p.p.). This finding was robust to adjustments for patient and procedure-level factors. The dose of midazolam, when administered, was not affected. Alert implementation was also associated with a decrease in the rate of midazolam administration for patients between 18 and 69 years undergoing general anaesthesia, despite the alert targeting older patients. Notably, our intervention included both an educational component and an EMR alert, which were rolled out in sequence. We did not observe a change in midazolam administration in the period that immediately following the educational component, but during which the EMR alert had not yet been activated. This argues that impacts of education alone are unlikely to explain our findings. As we did not separately study an EMR alert at a site that did not receive departmental education, we cannot separate the impact of the alert alone versus the alert combined with a preceding educational intervention.
This study adds to prior knowledge of how to reduce perioperative midazolam administration to older adults. Previous studies in perioperative settings have shown EMR-based nudges, such as visual reminders embedded into the intraoperative chart, can improve compliance with guidelinedirected patient management such as appropriate surgical antimicrobial prophylaxis. 14, 28 Donovan et al 11 demonstrated a bundle of interventions, including identification of cognitively vulnerable older adults 10 and implementation of default order-sets, can reduce administration of benzodiazepines though it is unclear which of their interventions drive this finding. We show that an EMR alert, preceded by clinician . Each data point represents (mg sum of all midazolam administered/number of cases in which midazolam was administered) calculated by month. Control group represents aggregate data from unexposed hospitals (the comparator arm). Exposed group represents data from intervention site. Grey bar over June 2019 represents the implementation period. Table 2 Study outcomes among patients aged 70 years and older. Results obtained from linear regression models adjusted for patient age, sex, race, ethnicity, American Society of Anesthesiologists physical status classification, body mass index, comorbidities (dementia, anxiety, cerebrovascular disease, cardiovascular disease, hypertension, chronic obstructive pulmonary disease, kidney disease, diabetes, peripheral vascular disease), use of volatile general anaesthetics (e.g., sevoflurane, isoflurane), case duration, and surgical department performing procedure. a Intervention effect is the difference-in-difference-estimate derived from the model. It is the coefficient of the interaction term between time period (pre versus post-intervention) and exposure included in the regression; b Includes patients who received at least one dose of midazolam intraoperatively. CI (confidence interval); p.p. (percentage point)

Outcome
Control facilities (95% CI) Exposed facility (95% CI) There are multiple possible mechanisms that could explain the impact of our intervention on care patterns. As a visual cue, the alert may have served as a "priming" stimulus to encourage guideline-concordant care. 28 Of note, the alert did not affect midazolam administration during cases performed under sedation where guidelines highlight its utility rather than dispensability. 27 The alert, which is displayed as a delirium prevention effort, may have also added salience to decisions about midazolam administration by eliciting memories of patients with delirium. 13 By establishing a norm of behaviour, the alert and the education session that preceded it may have drawn on established mechanisms for standardizing care practices. 29e31 Overall, the alert may have influenced individual providers to administer midazolam less frequently, contributing to our observation of decreased midazolam administration beyond the targeted population. Other types of nudges previously shown to influence anaesthesia provider behaviours, such as changing default options 32 and feedback tools 16 were not employed in this initiative but represent potential areas for future study.
Our study has certain limitations. This quality improvement initiative took place at a major academic health system, and specifically at a hospital with relatively low rates of midazolam use. Baseline patterns of midazolam use at each hospital were not known during the design and implementation study phases and were only discovered during retrospective analysis.   The generalizability of our findings to hospitals with different patterns of midazolam use remains to be determined. Reassuringly, the alert led to reduced rates of midazolam administration across surgical departments regardless of variable baseline rates of utilization. It is possible that our model failed to account for unobserved factors influencing midazolam usage patterns. For example, the final four months of our study occurred during the first wave of the COVID-19 pandemic, when case and patient mix varied considerably. Likewise, we did not consider variation amongst individual anaesthesia providers. These concerns are somewhat alleviated by the consistent results we obtained with analyses using time and group fixed effects to control for unobserved confounding. While we did not see a major impact of educational outreach in this study, education can powerfully alter provider behaviour, and a stronger outreach effort may have augmented observed effects. 33 The link between perioperative administration of benzodiazepines and postoperative morbidity remains an area of active investigation, 1,2 with recent studies reporting a single dose of preoperative midazolam to not significantly impact outcomes for older surgical patients. 34,35 While our intervention was based on best available evidence, the science and guidelines surrounding this topic is continually evolving. This study was not designed to measure impact on patient outcomes. Future prospective work may consider assessing the end-impact of EMR alerts on outcomes attributable to perioperative medications such as midazolam. Despite these limitations, our work is important for policy and practice. Among older surgical patients, receipt of potentially avoidable medications such as midazolam has been associated with adverse outcomes, including prolonged hospitalization, and guidelines recommend avoidance in this population. 3,5,6 Nevertheless, midazolam is commonly administered to older patients presenting for surgery. 8,9 Our findings suggest that an EMR alert, combined with education, can nudge anaesthesia providers away from administering midazolam to older adults perioperatively. Importantly, they also show that these alerts can substantially modify care beyond targeted populations.
In conclusion, an automated EMR alert directed toward anaesthesia providers and preceded by provider education and outreach, was associated with a decrease in perioperative midazolam administration to patients of all ages undergoing surgery. Further research is needed to determine the generalizability of findings and effects on patient outcomes. EMR nudges combined with education may be a useful strategy to reduce administration of benzodiazepines to older surgical patients.

Details of authors' contributions
All authors (SS, MC, SM, GS, RK, MN) made substantial contribution to conception and design, acquisition of data, or analysis and interpretation of data; drafting the article or revising it critically for important intellectual content; approve the version to be published; and agree to be accountable for all aspects of the work thereby ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Declarations of interest
The authors declare that they have no conflicts of interest.